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ABSTRACT 

In an instructional message the contextual dominance 
is most often conveyed in the form of printed or spoken sentences* 
Within any sentence used in conjunction with a picture are nouns or 
phrases that directly relate to contextual elements within the 
picture* These are called referents since they refer to objects 
perceptible in the picture* This study varied the dominance of 
referents used in a number of sentences and compared the patterns of 
subsequent observations of 15 pictures* The goal was to identify 
structural and/or contextual elements that stimulated consistent 
patterns of observation* Subjects were 21 students in a graduate 
class in research design at Indiana University* Thirteen subjects 
across 2 eyperimental groups received events with the same sequence 
of images, the verbal context being the only difference between 
groups* The other eight subjects made up a third control group 
exposed to images only* The study illustrated that when the 
structural dominance of an image is strong, an individual viewing 
this image will initially attend to the elements contributing to 
structural strength rather than to those which are contextual* 
(Contains 143 references*) (AEF) 
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The problem with pictures is that they are largely contextually defined instead of being structurally defined. 

That is, it is much more common for a person to describe a picture in terms of the contextual objects (people, places, or 
things)’within the picture than in terms of the visual elements that make-up the picture (circles, squares, points, lines, 
colors and shading). One might say, “Of course! It would be too cumbersome to describe a picture in those terms 
because they are not specific enough!’’. This statement is true if you are only considering the contextual meaning of a 
picture. Circles, lines, and points, have no contextual meaning of an by themselves. However, if an artist wishes to 
create or modify a picture, the only elements available for this purpose are lines, circles, squares, color and shading. 

There are no art brushes that paint faces automatically such that when the brush is dragged across the canvas a face 
appears. Instead the artist must vary lines, shapes and points, combined with shading and textures, in such a way as to 
produce contextually recognizable elements in the picture. The question is then, how does and artist know how to vary 
these structural elements in such a way as to produce the desired contextual meaning of a picture if these elements have 
no contextual meaning of and by themselves? Even more problematic is how an instructional designer specifies a 
necessary picture to an artist such that the appropriate contextual meaning is communicated fium the picture the artist 
creates 

In an instructional message there are often subtle, yet critical, aspects that must also be emphasized in 
supportive graphics or images. The instructional designer must be able to describe the contextual emphasis such that an 
artist or graphic designer will be able to manipulate the structural elements of the picture in a corresponding fashion. The 
goal is to make the structural emphasis congruent with the contextual emphasis. The emphasis of the contextual 
message and the emphasis of the structure of a picture is referred to in this paper as contextual dominance and structural 

dominance respectively. . . r r ■ j t. 

In an instructional message the contextual dominance is most often conveyed in the form of printed or spoken 
sentences. Within any sentence used in conjunction with a picture are nouns or phrases that directly relate to contexmal 
elements within the picture. For instance if we are presented a picture and heard “The man walked through the door.’’ we 
would expect to identify a man and a door in the picture. Both “man” and “door” may be called referents in the sentence 
since they refer to objects perceptible in the picture. The study described herein varied the dominance of referents used in 
a number of sentences and compared the patterns of subsequent observations of 15 pictures. The goal was to identify 
structural and/or contextual elements that stimulated consistent patterns of observation. A brief discussion of the 
literature, the methodology used, results for two pictures, and a summary of conclusions follows. 



Synthesis of the Literature 

Discussing how someone views and image depends also on describing the image itself. Most of the research to 
date use contextual definitions to describe the images and not structural ones; but, there have been some attempts to 
quantify the structure in terms of variations of color, complexity and layout. 



Color, Complexity, and Layout 

In general color has been found to be helpful when utilized to emphasize dominant contextual features, but it is 
a distracter when not relevant to context (Luder & Barber, 1984; Yarbus, 1967; Reid & Miller, 1980) For instance, 

Luder and Barber found that color could be utilized to highlight specific elements within a complex display, while Reid 
and Miller found that full color illustrations of anatomical drawings presented too many distracters from what was 
considered important in the drawing. 

Complexity and picture layout has been found to elicit different viewing strategies among students. 

Specifically^ Malcolm Fleming utilized four ”layouts’\ one with a caption preceding an illustration and another with it 
following. Each of these layouts also varied by complexity. Utilizing eye movement as a dependent vanable, he found 
that gender and previous experience with the information were strong factors as to which strategy to process the 
information was chosen— image first or caption first (Fleming* 1984). 

Another attribute* which relates to complexity* is that of the "degree of realism" to which an image may be 
attributed. The "degree" represents a continuum from concrete* or realistic* to abstract* or non-real istic. Gavriel Salomon 
describes this continuum operationally in terms of the degree of "coding" one must do when encountering a visual 
representation of something. 
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"Certain representations appear to be more "realistic" because their symbolic form comes closer to the 
way users represent the depicted entity to themselves. The less recoding something requires, the 
mentally "easier" it is to experience and the more "real" it appears. " (Salomon, 1981 pg. 201 ) 

Relative to this "concrete to abstract continuum", concrete images are remembered better than abstract images (Paivio, 
1983; Winn. 1982; Findahl & Hoijer. 1976; Wolf, 1970). On the other hand, abstract graphics have been found to be 
more successful in educational contexts due to fewer structural elements (Heuvelman, 1987). 

Visual Design of an Image 

One aspect left largely unstudied in most of the research in image utilization is that of the visual design of the 
stimuli being used. Visual design here refers to the structural relationships of visual elements within an image. Layout, 
mentioned earlier in the Fleming study of complexity, is the closest variable this researcher found in the literature to that 
of visual design. Nesbit captured this very common problem in the following statement. 

"Little attempt was made to examine the design qualities of the picture itself or the component cues of 
the picture in terms of learning theory. All modes of pictorial representation were considered as 
infinitely large masses of stimuli and examined as such... no attention was given to isolating those 
elements which make an instructionally effective visual." (Nesbit, 1978 pg. 496) 

The variable of style was of interest to Molner. He believed that traditional definitions of style, usually treated as purely 
affective and subjective, were identifiable in terms of structural attributes of an image. His eye movement study of 
Renaissance and Baroque paintings illustrates that the structural attributes of paintings may be utilized in such a stylistic 
manner that it can influence the viewer to scan an image at a certain speed and focus. He found that art of the 
Renaissance was viewed with large and slow eye movements, while Baroque art produced denser and shorter eye 
movements (Molner. 1981). 

Image and Language Processing in a Context 

Utilizing a "dual-coding information processing" model of learning, Kozma points out that if information in 
long-term memory may be stored, not only semantically but pictorially as well, then images can be retrieved into short- 
term memory in response to either nonverbal or verbal stimuli. He summarizes the research by stating that if the same 
information is stored both pictorially and verbally, it is more likely to be retrieved (Kozma, 1986). 

Just as printed text under an image provides contextual information utilized to view an image, so does spoken 
text during or immediately preceding an image affect the processing of that image. Many media today entail the learner 
processing audible textual information and visual information arriving at the same time. Very few studies have examined 
the intricate interactions involving the chunking, sequencing, and pacing of these dual-channel stimuli. But, it has been 
confirmed that the majority of the contextual effect is provided by the verbal text, and when verbal contextual cues are 
presented prior to viewing an image, a semantic attention to the image is evoked (Koroscik, Desmond. & Brandon. 

1985). 

This strategic focus supports previous prescriptive conclusions relating to color, complexity, and the schematic 
design of an image. Both point to contextual cues dictating structural prescriptions in the visual channel and specific 
reliance on the verbal channel for being the major carrier of contextual information. 

Koroscik, Desmond, and Brandon examined this relationship among structural, semmtic, and verbal contextual 
information. They began the study with the following hypotheses: It was speculated that the encoding and retention of 
art is subject to the type of contextual information given to viewers at presentation. Verbal labels with literal references 
to the objects, persons, or events depicted in an artwork ought to evoke semantic encodings that differ from those 
generated in response to verbal references pertaining to the work’s expressive qualities and/or other non-literal aspects of 
the depicted content. (Koroscik. 1984 pg. 332) 

By presenting first a verbal contextual cue and then presenting an image, they were able to measure the effect of 
the context on retention of the desired message. Since the contextual cues referred literally to structural elements within 
the images, some information was also gained about the degree of distraction of "non-pertinent" structural elements. 
Results also indicated that accurate interpretation of meaning was a function of the level of abstraction that characterized 
each artwork and of the type of contextual information given at input (Koroscik, 1984). 
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These results offer very few prescriptions to a message designer other than a sense that context and the structure 
of an image may somehow be interdependent. On one hand structural concerns are important for immediate interpretation 
and long-term memory, yet on the other there is, at some point, a shift of focus to contextual aspects of an image. 

Which is dominant, structure or context? What is the relationship between the abstract-to-concrete word continuum and 
an abstract-to-concrete image continuum? The concept of congruency begins to explore the relationships posed by these 
questions. 

Congruency Between Images and Language 

In his review of the contributions of eye movement studies to research, Marschalek states: 

..thus the compositional structure affects perception most dramatically when structure and meaning are 
united within the same areas of the pictu re. {Marschalek, 1986 pg. 135) 

This succinct statement encapsulates the idea of congruency between words and images. The concept of congruency 
between the structural components of an image and the context presented is of extreme importance to the message 
designer. Congruency deals with the basic problem of linking words and images together. Many researchers have found 
that when structural features and contextual features are congruent, then attention to the message is maximized 
(Heuvelman, 1987; Hsia, 1977; Marschalek, 1986; Miller, 1982; Nodine, 1982; Wember, 1976) 

A growing number of cognitive scientists are specifically looking at the relationship between the cognitive processing of 
words and the processing of images. A Dual-Coding Theory was developed by Paivio which stated that: 

"The verbalization of a picture's features increases the probability that two codes are activated in the 
formation of stimulus memories. The argument is that sensory features of pictures are stored in 
imagery codes, while the products of verbalization are retained as verbal or linguistic codes. " (Madigan, 
1982, pg. 80) 

This statement is contrary to a "common code" view that says pictures somehow possess a faster access than words to a 
common conceptual system. The dual-coding view, on the other hand, sees picture-word latency differences as stemming 
from a time consuming translation from one symbolic code to another and that semantic information required in the 
decision task is typically stored nonverbally. Numbers of other researchers have been testing similar concepts and have 
arrived at some meaningful conclusions. Segal and Fusella (1970) found, by using interference tests, that cognitive 
processing is modality (visual or audio) specific, i.e., the brain operates with a separate processing system for each 
modality. Nugent (1982) and Wickens (1984) found that "learners process pictorial and linguistic information through 
functionally independent, though interconnected, cognitive systems." 

Whenever one deals with issues of context, they are subject to personal interpretation as to their meaning, 
importance, and dominance. For an individual, attribution of meamng will depend on knowledge the viewer already has, 
knowledge that can be associated with the incoming information (Heuvelman, 1987). This view is commonly held by 
many researchers and recent language comprehension studies have indicated that language processing involves a context- 
dependent knowledge base that operates in an integrative and elaborative manner (Anderson & Oitony, 1975; Barclay, 
1973; Bransford, Barclay, & Franks, 1972; Marschark & Pavio, 1977). Dillen (1983), Braden & Walker (1980) and Wise 
(1982) also point to prior knowledge of the individual as a variable which significantly determines how an image will be 
perceived. Craik and Lockhart place prior experience as a comparative referent within the memory storage system. These 
researchers go on to describe two stages of the memory formation process. They are: 

EARLY STAGES... the analysis of physical features of incoming information...and 

LATER STAGES... concerned with matching input against stored knowledge from 

past learning, and with abstracting meaning." 

Some researchers have consistently found that when images are utilized which include people, fixations center on their 
faces to the almost total exclusion of anything else in the image (Buswell, 1935; Chu & Schramm, 1975; Guba, et al, 
1964; Yarbus, 1967). The implication of this is that people know from experience that faces and animate objects, in 
general, are primary providers of contextual information. 
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Animate objects receive a higher density of fixations than inanimate objects when both are contained in the 
same picture. When considering portraits, the highest density of fixations occurs on the eyes, nose, and mouth because 
these areas of the face tend to convey information concerning emotion and the degree of physical attractiveness of the 
individual. (Yarbus, 1967 pg. 28) 



Structural and Contextual Dominance 



A primary concept within this study, in relation to the analysis of contextual and image structure, is that of 
dominance. It has been found that it is possible to define an image in terms of its physical structure apart from its 
contextual elements (Friedman & Poison, 1989; Koroscik, 1984; Marschalek, 1986). Likewise it is proposed that 
linguistic analysis can focus on both the structural aspects of a sentence apart from the contextual aspects (Mason, 
Kniseley, & Kendall, 1979; Smith & van Kleeck, 1986). In each of these four categories it is proposed that a dominance 
exists which may be utilized in comparing these categories, resulting in a description of congruence, non-congruence, or 
ambiguity. It is toward the definition of image structure, and specifically those structural elements which affect 
dominance, that this study is directed. 

Since no models emerge from the literature, we may turn to the discipline of graphic design for a description of 
visual form by Wucius Wong (Wong, 1972, pg. 6) which seems to fit a hierarchical description of visual form. An 
adaptation of his hierarchy appears below. 



Visual Form: 

as BASIC ELEMENTS 
as STRUCTURAL ELEMENTS 
as RELATIONAL ELEMENTS 
as CONTEXTUAL ELEMENTS 



of Point, Line, Plane, Volume 
of Shape, Size, Color, Texture 
of Contrast, Direction, Position, Space, Gravity 
of Representation, Function, Meaning 



These 4 levels of visual form give us not only a language to describe what we sec in an image, but also a hierarchy 
which is harmonious with our previously stated levels of analysis. Describing visual form in this manner is no simple 
task, for in order to compare images reliably they must be described in terms of their attributes at each of these four 
levels. It is not long before one realizes that in fact it indeed takes a thousand words to describe a picture. This 
overwhel'iing task of image description is reduced through the use of the concept of dominance. 

In discussing the structural form of an image, the most appropriate discipline to draw from is art criticism or art history. 
Specialists in this field are skilled in using words to describe images. A common practice of many art historians is to 
reduce what may seem to be a very complex image to a few general shapes, colors and textures. An example of this is a 
description of Goya's "The 3rd of May 1 808; The Execution of the Defenders of Madrid" (Fig. 1 ) 
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Figure 1 




Its organic structure, based on triangles and strong diagonals, is peculiarly fitting to the theme, and its 
neutral colors in grays and browns, with a splash of red in the pool of blood heightens its emotional 
impact" (Gardner, 1959, pg. 443) 

One item to note from this description is the implication that these structural elements are arranged to produce 
an emotional impact and not a linguistic one. This underscores the dual-coding caveat of images being processed in the 
affective domain of cognition. This description is very brief and encapsulates hundreds of individual elements into gross 
generalizations. It is necessary to exclude lesser elements if we must focus on the dominant ones. It is interesting to 
note that even this brief description alludes to contextual elements of the "theme" and the "pool of blood" and is 
obviously formed as a caption intended to be read while viewing the painting. This underscores the difficulty in 
separating contextual elements from structural ones. 

There are hundreds of individual structural elements including at least 22 people, 16 faces, all of their wearing 
apparel, weapons, the foreground elements, the city in the background and the dark night sky. That we can attribute 
meaning to the shapes and brush strokes alerts us to the fact that we are using contextual descriptions of structural 
elements. If we use entirely contextual descriptions Goya's painting becomes much simpler. We could describe it as an 
image consisting of three groups of people in front of a wall with a city in the background. Another description of the 
same work is presented to illustrate almost an entirely contextual description. 

"Here the blazing color, broad fluid brush work, and dramatic nocturnal light are more emphatically 
Neo-Baroque than ever. The picture has all the emotional intensity of religious art, but these martyrs 
are dying for liberty, not the kingdom of Heaven; and their executioners are not the agents of Satan but 
of political tyranny - a formation of faceless automatons, impervious to their victims' despair and 
defiance. "(Janson, 1965, pg. 479) 

It would seem to this researcher that Goya would support both this description and Gardner's earlier one because 
he chose structural elements and manipulated them in such a way that the later contextual description would be perceived. 
He chose to manipulate the structural elements of grays and. browns so that the red pool of blood would be dominant. 

The two overlapping triangular shapes point like arrows toward the two dominant groups of people. We view the scene 
just an instant before the pure, bright white shirt of the rebel is to become crimson from the lines of the rifles pointing 
directly at it. In both examples stated above, the structural features of the image are tied directly to the contextual 
features. The artists have selected and arranged structural elements in such a masterful way that the desired context is 
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effectively communicated to the observer. Another way to state this is that there is congruence between the structural 
dominance of the image and the contextual dominance that was the intent to communicate. 

Based on the findings in the literature and translated into the terms of dominance, context, and congruency, the 
following hypothesis were adopted for this study and which the methodology was designed to test. 

Hj. Congruency between structurally dominant elements and contextually dominant elements will result in 
greater attention to these elements than in less-congruent situations. 

H 2 . As the complexity of an image increases, the structural dominance will decrease. 

Hj. As structural dominance decreases, attention will be directed more to c *)ntextual elements. 

Methodology 

Subjects 

A total of 21 subjects were used in this study drawn from a graduate class in research design at Indiana 
University. 13 subjects across two experimental groups received events with the same sequence of images, the verbal 
context being the only difference between groups. 8 subjects made-up a third control group exposed to images only, 

Definitio!! of Variables 

The Contextual Dominance Variable 

he verbal contextual dominance variable was defined as a spoken phrase immediately preceding exposure to an 
image. For each stimulus event, contextual dominance was derived from the nouns within each phrase and then given 
points that were interpreted in terms of the image. Due to the need to compare the verbal contextual variable to the 
image structural variable, the contextual dominance was described in terms of the referents appearing in the image sectors. 

The Ima^e Structural Dominance Variable 

The image was described in terms of two criteria, structural dominance and degree of complexity. The first level 
of image description, or structural dominance, was defined in terms of prioritized sectors of the image as assigned by a 
panel of visual design experts. 

The Image Complexity Variable 

The second level of analysis was that of complexity. Complexity was determined simply be tabulating the 
number of structural elements visible in the image, e.g. 10 lines, 20 circles, 6 shapes, etc. In this study images were 
grouped into simple, medium, and complex categories. The criteria for inclusion in the simple category was having less 
than 10 structural elements, in the medium category by having more than 10, but less than 50 elements, and in the 
complex category by having over 50 structural elements in the image. 

The Congruency Variable 

Identifying the degree of congruency between two groups was a statistical process of shared dominance 
comparisons. Based on the Experts' rankings of structural dominance, phrases were constructed which either referred to 
this dominance or referred to less dominant sectors of the image. Congruency was defined as an independent variable with 
two conditions. A "Congruent Condition" exist d when the contextual phrase that was heard referred to elements 
identified as part of the image structural dominance. The phrase which referred to less dominant elements within the 
image was assigned to the non-congruent condition. 
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MateHsls 



Fifteen images were divided into three groups of five according to the above complexity criteria. Structural 
dominance was determined and validated by a panel of visual design experts. Two contextual phrases we written for 
each image with one phrase written to match the structural dominance of the image and the seevnd phrase focusing on 
weaker structural elements of the image. 

A sample from each of the three levels of complexity follow: 

Simple Complexity 



Fig 2 (Image #5) 




Image Structure description 



Shapes: 

Colors: 

Textures: 

Relational: 



2 circular shapes (1/5 width diameter) 
black and blue on white background 
solid (no textures) 

diagonally positioned with the blue circle in the 
upper/Ieft ninth of the frame and the black circle 
in the lower/right ninth of the frame. 

Contextual Phrases 



Congruent: Since blue knights are instinctively afraid of black 

dragons, he ran. 

Non-Congruent: Since black knights are instinctively afraid of blue 

dragons, he ran. 



Moderate Complexity 



Fig 3 (Image #8) 




Image Structure description 



Shapes: 

Colors: 

Textures: 

Relational: 



1 hollow triangle shape made of moderately thick lines, 
black on white background 
solid (no textures) 

Isosceles triangle fills over 1/3 of frame 
and is centered in frame. 
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Contextual Phrases 



Congruent: A pyramid points to the sky. 

Non-Congruent: A pyramid is heaviest at its base. 

Complex Complexity 



Fig 4 (image #14) 
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Image Structure description 



Shapes: 

Colors: 

Textures: 

Relational: 



lines and a variety of shapes but mostly rectangles, 

1/4 sphere, circle sections. 

grayish beige, bluish green, white, yellow. 

rough (stone-like), and slick 

synmietrical 3-dimensional space, 

white object at bottom, illustration at top of image. 

Contextual Phrases 



Congruent: The center alters were usually simple 

with only two candles and a cross. 

Non-Congruent: The biblical phrase “The Lord is my shepherd” 

is reflected in the decorations around the half dome. 
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Procedure 



Gathering E\e-Movement Data Fiu 5 




Subjects were asked to sit in a specialized ‘ head-stabilization” chair (Fig. 5) A small light was place adjacent to 

their right eye and a video camera was trained and focused on the eye ball and a portion of the light. The 

strategy was to record the reflection of the light on the cornea of the subject relative to the stationary light bulb. 

Following a registration process, the stimulus materials were presented. 

Recording E\e-Movemeni Data 

The videotape was subsequently played-back one frame at a time and the motion of the reflection of the light off 

of the subject’s cornea was documented s being in one of nine sectors of the frame. 

Statistical Analysis 

The 'Within^Group analysis^ 

Groups A, B, C, and the Expert Group were included in the analysis for each of the 15 images. All data were 
converted to percentages so as to allow for a direct comparison between the stimulus events and the Expert rankings. A 
Kendall Coefficient of Concordance (W ) was computed for each group associated with each image. This resulted in a 
rank-ordering of the means of fixations for each sector of the image, a Chi-Square, and a significance value for each 
group. If the result of the Kendall test was significard, then a more rigorous test of significance, the Friedman Two-Way 
Analysis of Variance by Ranks, was calculated. If significance was acceptable, calculated by the Friedman test, then a 
multiple comparison between individual rank-order means was calculated to determine the significance of a sector’s 
dominance (Siegal, 1988). 

Analysis of E ve-Movement Data 

The raw data, which consisted of an ASCII file of 75 data pairs, 1,C 2,C 3,B 4,E ... 75,H , were read into a 
HypeiCard Stack and sorted into the total number of fixations registered in each of the nine sectors. 

Determining Dominance from Rank-Order Comparisons 

Dominance for experimental data (that which was derived from eye movement data), as well as the ranking by 
the experts, is defined as those sectors which produced significance from the Comparison of the Means. These sectors are 
then said to represent 100% for the Dominance for that experimental group of data and are discussed in terms of the 
percentage which each sector claimed of that dominance. For instance, in the example below(Fig. 6), the dominance for 
the congruent condition of image 8 (Sectors B and E) is 89% and 1 1% respectively. 
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Figure 6 




-mining Contextual Dominance 

Contextual dominance was determined by identifying the nouns within the contextual phrases and assigning points in 
relation to: 

1. their position within the sentence, 

2. the presence of visual modifiers that were also in the image, and 

3. whether they were a focus of attention* within the sentence. 

The points accumulated for each noun were then assigned to the sectors of the image to which the noun referred, The 
sectors to which the nouns referred represented 100% of the contextual dominance and were represented in terms of the 
percentage which each sector claimed of the dominance. 



The Between»Group Analysis 
Congruency 

A “Congruent Group" was defined experimentally as a group which received an audible contextual stimulus 
which contained a referent to a structurally dominant element within the image. The assignment of an experimental 
group to either the Congruent data group (CG-S) or the Non-Congruent data group (NCG-S) was based on a strategic 
decision by the experimenter to match the contextual referents heard verbally with the structurally dominant attributes 
within the image. 

The Degree of Congruency (°CG) was a post-hoc statistic used to describe the relationship between data groups 
in terms of the overall similarity of their shared dominance. This statistic produced a number, from 0 to 100, with the 
most congruent condition being 100. This procedure was executed as follows (see Fig. 7): 

Corresponding sectors within each of the two data groups being compared were sequentially added together producing a 
"Sector Sum". Also these same numbers were subtracted from each other to produce a "Sector Difference", The process 
was repeated for each sector as long as there were values present within the sectors being compared. The sector sums and 
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the sector differences were totaled, then the difference between these totals was found, and this figure was divided by 2 to 
produce the final result. 

The formula for determining the Degree of Congruency (°CG) was as follows: 

(Kseclor sum) - L(sector difference)) + 2 = °CG 



Figure 7 




% DOMINANCE % DOMINANCE 



For example, comparing the EXP group with the CG-S group in Figure 7: 

38+89=127 & 31+11=42 
Z(Scctor Sum)= 127442= 169 

89-38=51 & 31-11=20 
^(Sector Differen.ce)=5 1 +20=7 1 

( 169-71 K2= 98+2= 49 = °CG 

Determination of Structural & Contextual Effects 

In order to examine the effects of the structural dominance and contextual dominance on the Congruent and Non- 
Congruent data groups, the sector of primary dominance of each was compared. The percentage of the dominance accrued 
in the sector was considered to be representative of the contextual effect for that group. The same procedure was followed 
for the Non-Congruent data group. 

A similar approach was utilized to determine the structural effect. The sector of primary dominance for the 
Expert data group determined which sector of the Congruent or Non-Congruent data group represented the structural effect 
for each group. 

The Effects across Complexity 

The images were grouped into 3 levels of complexity - Simple (images 1-5), Moderately Complex (images 6- 
10), and Complex (images 11-15). By plotting the contextual effects for each image, an examination of the interaction 
effects of complexity was undertaken. 
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Results 



Comparison of Groups by Image 
Data Group Definition 



Since treatments for congruency were randomized among experimental groups A & B, and later recoded into 
groups of congruency, groups A & B do not appear as units in the results section. Instead, each stimulus event (the 
presentation of an image, with or without a specific phrase) is identified as being either congruent or non-congruent. 
These events are coded as congruent or non-congruent depending on whether or not the cor^textual dominance of the verbal 
phrase matches the structural dominance as defined by the Experts. This recoding establishes 6 groups for comparison. 



Experimentally defined: 
(EXP) 

(FR-S) 

(CG-S) 

(NCG-S) 

Theoretically defined: 
(CG-X) 
(NCG-X) 



Ranking-Structural Definition / Expert data group 
Eye Fixations / Free-Viewing data group 
Eye Fixations / Congruent data group 
Eye Fixations / Non-Congruent data group 

Contextual Definition / Congruent Condition 
Contextual Definition / Non-Congruent Condition 



Within Image Results 

Presented here are the results of one image from each of two complexity groups. Individual within-group data 
are presented graphically via a nine-sector rectangle containing the resulting percentage of dominance indicated in the 
appropriate sector. The sector which is primarily dominant is outlined with a solid box. The Kendall W and the 
Friedman test results are presented adjacent to each data group's dominance rectangle. The Degree of Congruency is 
indicated in a box joined by two lines connecting the groups being compared. 

After the presentation of the results of each of 2 images, will be a discussion of the results across the 15 images 
by complexity. This comparison will utilize only the Kendall W and the primary sector that relates to the structural 
and/or contextual dominance. 



A • B • C 




a • H J I 



Definition 



Image #8 



Contextual 

Congruent 

A pyramid points to the sky. 

[N] [N][AN][LN] 

1B4E4G4H41 3A,3B,3C 



Non-Congruent 

A pyramid is heaviest at its base. 

[N] IN][AN][LN] 

1B4E4G4H41 3G,3H4I 
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Eye Movement Results 



Kendall W. Friedman: 

Significance, Dominance, & Degree of Congruency 




Verbal Summary for Image #8 

Stralefies and Anticipated Results 

From a visual convention standpoint, a triangle is one of the three basic graphic shapes (the others are a circle 
and a square). This shape also is utilized as the head of an arrow which implies direction toward the tip (which would 
emphasize sector B if structural dominance is strong). The triangle was renders! in outline form so as to highlight the 
space inside the shape as well (sector E). The contextual cues were designed to emphasize the top in the congruent 
condition and the bottom in the non-congruent condition in the anticipation that fixations would dominate these sectors if 
there was a strong contextual dominance effect. 

General Stimulus Event Comparisons 

Ranking of sector preference was significant beyond the .01 level for all data groups. Uniformity of image 
processing was moderate for the Expert data group, as well as for all experimental groups. 

•^(EXP) =.5736 & y/(FR-S)(CG-S)(NCG-S) of <.6884 

Contruen rv Comparisons 

The Degree of Congruency (°CG) between what the Experts identified as structural strengths in the image and 
how the Free- Viewing data group viewed the image was moderately strong, °CG(^eXP\FR-S) =81- The match between 
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the Free-Viewing data group and the Congruent data group was moderate, °CG(fn.s\CGS) =54, and the match with 
the Non-Congruent data group was moderately strong, °CG( fuswCG-S) =72. 

Structural & Contextual Dominance Comparisons 

A very strong Congruency effect was evident in the Congntency data group with this image. Nearly 90% of all 
fixations were accumulated in sector B (HfifCG-Sj = 89%). The Structural effect, in the Non-Congruent data group, 
was strong enough to draw enough attention from the contextually dominant sector H producing a strong attention to 
sector B also (DB(NCG-S) = 43%). There was some contextual effect evident in the Non-Congruent data group, 
evidenced by attention to sector H (Df{(i\fCG-S) = 29%), compared to no significantly dominant fixations appearing in 
sector H for the Free-Viewing data group (Dfi(FR-S) = 0%). 



Image #14 



I 





[N][VN] 

2H 



[N] 

IH 



[N][LN] 

2H,2B 



Non-Congruent 

The Lord is my shepherd" is reflected in the decorations around the half dome. 



tN] 

IB 



[N] 

1A,1B,1C 



[N] 

lA, IB, 1C, ID, IE, IF 



[N][LN] 

2A,2B,2C 




Eye Movement Results 




Strategies and Anticip ated Results 

This image is the only "realistic" photograph of the 15 images. The picture was taken from the point-of-view 
of someone standing in the center of the room at an eye-level of the second alter. This angle accentuates the architectural 
perspective lines which converge just under the center window. These perspective lines alone were expected to draw 

strong fixations to sector E. j- • . 

The first "object" encountered in this image is the "floor alter", which is referred to in the congruent condition s 
contextual cue. A focus on sector H would reinforce a congruent condition. The non-congruent contextual cue refers to 
the mosaic illustration above the center window, which is the largest graphic element in the image, as well as the largest 
architectural element (the half-dome in sectors A.B. & C). Due to the complexity of this image, it was anticipated that 
the contextual effects would be strong predictors of the eye-fixations. 

General Stimulus Event Com parisons 



Ranking of sector preference was significant beyond the .01 level for all data groups. Uniformity of image 
processing was moderately high for the Non-Congruent data group, moderate for the Expert and Congruent data groups, 
and low for the Free-Viewing data group. 

•^(NCG-S) = 7590 & ^(EXPHCG-S) of < S11S& V/(FR-S) =-4633 
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Congruency Comparisons^ 



The Degree of Congruency (°CG) between what the Experts identified as structural strengths in the image and 
how the Free-Viewing data group viewed the image was high» °CG(£XP\FR-S) Th^t match between the Free- 
Viewing data group and the Congruent data group was high, "'CG(fK.s\CG-S) =82, and the match with the Non- 
Congruent data group was moderate, ^GG(fji^.s\NCG’S) =^^- 

Structural <k Contextual Dominance Comparisons 

A weak congruency effect was evident in the Congruent data group in sector H (Df{(CG-S) = 38%). A strong 
contextual effect was evident in the Non-Congruent data group in sector B (Db(NCG-S) = 40%). A definite focus on 
the center of the image was evident in the Free-Viewing data group (Pe(FR’S) = 43%). Similar to the preceding 
image, if in fact the actual structural dominance for image 14 is in sector E, as is evidenced by the Free-Viewing data 
group, then both the Congruent and Non-Congruent data groups demonstrate contextual effects rather than congruency 
effects. However, it may also be said that both groups represent partial congruency effects, since both sectors B and H 
represent secondary and tertiary structurally dominant sectors. 

Between Image Results 

The 15 image stimuli presented were grouped into 3 subgroups; 5 simple, 5 moderately complex, and 5 
complex images. The Kendall Coefficient of Concordance measures the uniformity of the subjects' responses. The 
Kendall W (W ) is a rating as to how similar each subject's response was to other responses such that if they were 
identical, the W would be 1 . Figure 8 illustrates the general decrease of uniformity of the experimental groups as the 
complexity of the image increased. 

Uniformity by Complexity 

The Experts (EXP) began with very strong agreement (.9) and. as the complexity of the image increased, their 
average uniformity dropped, but still remained in the strong moderate range (.6). The Free-Viev;ing Group (FR-S), on 
the other hand, began in the moderate range (.6) and finished with a low uniformity W (.42). The congruent and non- 
congruent conditions (CG-S & NCG-S) displayed lessening uniformity as complexity increased, but stayed in the 
moderate range. 

Comparisons by Primary Dominance 

By plotting the dominance of the contextually dominant sector of both the Congruent and Non-Congruent data 
groups, a comparison of the focus of these groups may be made (Figure 8). 
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Figure 8 



Contextual Focus Across Complexity 

by Primary Dominance Sector 

Conflroent Non-Congruent) 
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SIMPLE MEDIUM COMPLEX 



It is evident in this graph that the Congruent data group consistently exceeded the Non-Congruent data group's 
focus on the contextually important element of the image for the simple to medium complexity images. It is also 
apparent that with images 12, 13, 14, and 15 (within the Complex category), the Non-Congruent contextual dominance 
exceeded that of the Congruent data group. 

Another observation evident from the graph is the strong congruency effect for the moderately complex images. 
It was with the images dealing strongly with visual conventions that the Congruency Effect was most evident, e.g. that 
structural dominance, coupled with contextual dominance, had the greatest effect Contextual dominance alone 
contributed more with the complex images. 

Conclusions 



Response to the Specific Experimental Hypotheses 

[Hl]The first experimental hypothesis posed was that congruency between dominant structural elements and 
dominant contextual elements would result in greater attention to these congruent elements than in non-congruent 
situations. The statement, in terms of this experiment, looks specifically at the contextual dominance generated in each 
of the Congruency data groups. This dominance represents not only a contextual effect, but also the contribution of a 
structural effect as well. Because of the combined effect, it is called a congruency effect. In the Non-Congruent data 
group the dominance primarily represents the contribution of a contextual effect. In 69% of the cases, a focus on the 
contextually dominant element of the image was stronger when that element was also structurally dominant 
A Congruency Effect was particularly evident with the moderately complex images, where the average dominance was 
34% higher than that in the Non-Congruent condition. These results lead us to a conditional acceptance of the first 
hypothesis. If the structural dominance is strong, then congruency between structurally dominant elements and 
contextually dominant elements will result in greater attention to these elements than in less-congruent situations. 

One image was exemplary in demonstrating Congruency Effects ~ image #8. Reviewing the structure of this image and 
context will clarify the types of "strengths" referred to above. The word "sky", heard in one group's contextual phrase 
referred to three sectors. A, B, & C, which were clearly related to the upper third of the image. The other group heard the 
word "base", which refers to the lower third of the image. The action verb, "points," which was heard by the Congruent 
data group, coincides with the visual convention of an arrow (which a pyramid shares in shape) that reinforces the 
congruency between contextual cue and image structure. The fact that the object was an isosceles triangle rather than an 
equilateral one increased the "pointing-up" effect even more (DB(CG-S) =89%). The Non-Congnient data group was 
directed to the base of the triangle by the contextual cue, but they had to overcome the strong image structure "pointing- 
up" to attend to the base. A very small portion of their dominance was devoted to the base area of the image (DH(NCG- 
S)=29%). This image produced the greatest Congruency Effect. It also demonstrated a strong Structural Effect evident in 
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the inflation of sector B in the Non-Congruent data group (DB(NCG-S)=43%). There is little or no explanation for such 
dominant fixations to occur in this group except for the fact that the image structure was so strong as to draw fixations 
into sector B. 

[H2]The second experimental hypothesis stated that, as complexity of an image increases, the structural 
dominance will decrease. Across the 15 images there was a continual trend downward in uniformity of agreement of 
structural dominance by the Experts. These results in uniformity, and loss of agreement on structural dominance, 
support the second hypothesis. Specifically it provides evidence that, at least with these images, as elements of equal or 
greater structural value are added to an image, the structural dominance declines in strength. 

[H3]The third hypothesis was whether or not structural dominance is inversely proportional to the contextual 
effect; or, as structural dominance decreases, does the contextual effect have a greater impact on the attention patterns of 
the observer? With the information just presented on uniformity, if the hypothesis is true, then one would look for an 
increase in the contextual effect from the more complex images where uniformity was low, as was the structural 
dominance. 

Referring to the graph in Figure 8, this hypothesis is supported by the Non-Congruent data group's performance 
exceeding that of the Congruent data group for images 13 through 15. As was mentioned in the detailed description of 
the complex images, the structural dominance was so illusive for the Experts that the primary dominance they indicated 
for the complex image set never matched that of the Free-Viewing data group. This success rate may be compared to a 
match of 60% in the moderately complex image set (images 6 through 10). 

A case in point is that of image #14. One of the Experts described the way he thought that persons viewed a 
photograph was spatially (e.g. the same way one would if he were to be walking in that space). In other words, one 
would look at the first thing one would come to, then the second, then the elements surrounding one's immediate space, 
then beyond (in the case of image #14 this would produce an H, E, B progression of D ). Another Expert suggested that, 
since everything was arranged symbolically in a hierarchy from God, to Christ, to the sheep, and finally down to 
common man at the altar level, that the viewing patterns would coincide with that order (or a B, E, H progression of D ). 
In fact, the viewing was more determined by the context presented by the verbal cue. The progression of D was H, E, B, 
D for the group which received a contextual cue relating to the alters at the bottom and center of the image, and B, E, A, 

H for the group which heard reference to the illustration in the half dome. This supports the third hypothesis that, as the 
structural dominance decreases, the context becomes more important as to where one focuses attention. 

This finding would be supportive of the majority of research to date in the area of complexity, which states that 
simpler visuals are more predictable and memorable (Lx)ftus, 1972; Luder & Barber, 1984; Friedman & Liebelt, 1981 ; 
Palmer, 1975; Heuvelman, 1987). New information is offered from this experiment regarding the relationship of Expert 
predictions. Congruency and Free-Viewing situations in terms of complexity. These variables have not been compared in 
the past. 

The Concepts of Structural and Contextual Dominance 

A major contribution of this study is its identification of Structural and Contextual dominance as factors which 
define Congruency, accompanied by a methodology for measuring them. Many studies have identified the structure of an 
image as being a variable (Heuvelman, 1987; Marschalek, 1986; Koroseik, 1984; Nodine, 1982; Molner, 1981 Nesbit, 
1978; Dwyer, 1972), but none have defined it in terms of dominance. Many of these same researchers have presented a 
verbal or written phrase as a contextual setting to provide semantic encoding of the image, but none have compared 
contextual and structural dominance directly. 

To the visual designer, the structural aspects are of utmost importance, since these are the elements that are 
being manipulated. The sad part is that messages are designed without attention to the structural dominance, or visual 
design, and because of their incongruity with the message, are ineffective. Research is done without concern to this very 
strong interacting variable, and inconclusive results are the outcome. The sooner that structural dominance takes its place 
beside contextual dominance as an equal contributor to the message, the sooner we will begin unraveling many of our 
communication issues involving images. 

The Concepts of Congruency and Complexity 

The finding in this study that Congruency Effects were strongest in the moderate complexity range supports 
Heuvelman's and Koroseik's finding that congruency can be higher with more abstract images. This appears particularly 
true when familiar visual conventions are used at this level of complexity. This study only partially supports other 
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findings from these researchers that realistic or highly complex images distract to such a degree that congruency cannot be 
achieved. This study supports these views only when the stimulus consists of a weak contextual dominance coupled 
with a strong structural dominance. In these situations the dominance of the image takes over, and the contextual thrust 
is lost. However, this study also demonstrates that, especially in complex images, if the contextual dominance is strong 
and if the visual referents that are elements of the weaker structural dominance are also to by found within the image, 
then a contextual effect will also be strong, because of the contribution of a small congruency effect. 

Summary 

This study illustrates that when the structural dominance of an image is strong, an individual viewing this 
image will initially attend to the elements contributing to structural strength rather than to those which are contextual. 
The intent of most educators, in the presentation of an image, is mostly contextual. If a learner who rates low in 
knowledge of, and motivation toward, the specific information being presented, and if the image used is strongly 
structured in favor of a different context than that which the instructor intends, then the chances are high that the learner 
will not process the image in a desired manner. If this lack of congruency continues during a presentation, confusion on 
the part of the learner will be the most likely outcome. The converse of this situation is the desirable state of affairs. 
Instructional designers must gain adequate respect for the structural dominance issues, as the majority already have for the 
contextual dominance ones. The chances that learners encountering instructional messages will find the pathway to 
knowledge must greatly be enhanced. Most instructional designers do not have the skills necessary to analyze the 
structural components of an. image to determine the structural dominance. Graphic designers, and a newly forming role 
called an interface designer (mainly involved with electronic media production), do have these skills, but most often are 
not adequately informed in the subject matter or instructional strategies to determine the appropriate contextual 
dominance. A team effort is necessary to assure that both aspects of the instructional development are adequately 
pursued. The inherent problem is that one person talks context while the other talks structure, and no progress is made. 
This study opens the door for communication between these two important people utilizing the rubric of Dominance. If 
both the instructional designer and the interface designer respond to each others’ products in terms of the "dominance 
perceived,” not only will the dialog be more fruitful, but the learner (who will be the ultimate "perceiver") will also be 
brought into the process. 

Placement of the message along a quantitative to qualitative continuum becomes a negotiation item for the 
"team." The interface designer will inherently be an advocate for more qualitative messages at one end of the continuum, 
while the instructional designer most likely will support more quantitative solutions at the other. 'Villemmn (1966) 
supports the need for both in his statement that, "Qualitative mediations are instrumental to focally cognitive operations 
at one end of the spectrum, while on the other they become focal with cognitive elements assuming the instrumental 

role.” . . . . t- • j 

Major work to be done in research is in the field of art, not to quantify it and make it ngid, but instead to 

understand it adequately enough to incorporate it into the message design. The extent that anyone creating a message 
desires a level of perfection in communication, is the degree to which one embraces a desire to deal with the total 
continuum from information to art. Let John Dewey s challenge be our motivation. 

As long as art is the beauty parlor of civilization^ neither art nor civilization is secure.,, the ideal 
human community is dependent upon its esthetic component. Rather than envisioning art as an 
‘^efflorescence " , it is a condition of the realization of democracy, conceived as an ideal that lends a 
distinctive character to all aspects of life. If such a view is tenable, then significant revisions in 
democratic educational and social theory are in order. {Dewey, 1934) 
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